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ABSTRACT 


'  In  this  paper  we  consider  a  class  of  nonlinear  estimators  that  are  the 
solutions  to  a  certain  variational  problem.  These  estimators  generalize 
the  concept  of  conditional  expectation;  we  investigate  their  continuity  and 
convergence  properties. 


I.  INTRODUCTION 


In  this  paper  we  will  develop  some  machinery  that  generalizes  some 
well  known  approximation  and  continuity  properties  of  minimum  mean  square 
error  analysis.  These  will  allow  us  to  examine  estimation  theory  in  a  much 
more  general  context  and  give  us  a  much  wider  choice  of  criteria  for 
analyzing  and  penalizing  error.  We  will  be  required  to  forego  some  of  the 
main  convenient  accoutrements  of  minimum  mean  square  error  analysis:  the 

2 

Hilbert  space  structure  of  L  ,  the  projection  theorem,  and  the  fact  that 
minimum  norm  projection  onto  a  closed  subspace  is  a  nonexpansive  linear 
operator.  However,  in  return  we  receive  a  method  that  allows  us  to 
customize  our  notion  of  error  to  our  particular  model.  Oftentimes,  mean 
square  error  is  not  the  appropriate  fidelity  criterion. 


II.  PRELIMINARIES 

Q  Throughout  this  paper  let  (Sl.^.P)  denote  a  probability  space  and  let 
Lu(fl,^,P)  denote  the  set  of  all  random  variables  X:ft  -*  IR  modulo  a.s. 
equivalence  equipped  with  the  topology  of  convergence  in  probability. 

Let  <&:  [0,°°)  +[0,°°)  be  convex,  increasing,  and  satisfy  4>(0)=0.  We 
define  the  Orlicz  space 

L*(ft,£\P)  *  {XeL°(Q,5(’,P):  /  *(|X|)dP<»}. 

The  function  4>  is  said  to  satisfy  the  doubl i ng  condition  if  there  exist 
C,M  >0  such  that  x  >M  =>  <J>(2x)  < C4>(x) .  If  $  satisfies  the  doubling  condi¬ 
tion,  L^(n,^,P)  is  a  vector  space.  Moreover,  it  is  a  Banach  space  with 
the  Luxemburg  norm 

||X||  #  =  inf  {X  >0:  /  4*  (i^- )  dP  <  4>(1)}. 

Note  that  this  is  simply  the  Minkowski^functional  of  a  certain  subset  of  L  . 
Also,  if  {X  ,  and  X  are  in  L*,  X„  ^-*-X  if  and  only  if  lim  f  $( |X  -X| )dP 

For  the  basic  facts  on  Orlicz  spaces  see  [3].  Details  on  Minkowski 
functions  can  be  found  in  [1],  pp.  294-295  or  [6],  pp.  23-26. 

Let  t:  [0,«>)  -►[0,o°)  be  increasing  and  have  a  strictly  increasing  first 
derivative  <p  with  <p(x)  ■*■«>  as  x  +°°.  Since  <f>:  [0,°°)  -*-[0,°°)  is  a  homeomor- 


Preeented  at  the  Twenty -Third  Annual  Allerton  Conference  on  Communication , 
Control,  and  Computings  October  2-4 3  1985;  to  be  published  in  the 
Proceedings  of  the  Conference. 
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phism,  ^=(j)“  is  also  continuous,  strictly  Increasing,  and  satisfies 
11m  iij(x)  =  «.  We  define  the  conjugate  function  f  to  4  to  be 
x-°°  x 

f(x)  B  /  <Ht)dt,  x>0. 

0 

This  Is  somewhat  more  restrictive  than  the  definition  employed  In  [3],  but 
It  eliminates  certain  annoying  technicalities.  It  also  allows  us  to 
dispense  with  the  customary  nonatomicity  assumption  made  about  the  measure 
spaces  used  In  this  context.  We  call  the  pair  $,4'  an  Orllcz  pair. 

The  Banach  space  L$  Is  reflexive  If  and  only  If  4*  and  <4  satisfy  the 
doubling  condition.  A  simple  calculation  shows  that  the  strict  convexity 

of  <&  implies  that  of  the  Luxemburg  norm  on  L*  and  that  of  the  convex  func¬ 


tional  X  -*  /  4>{|X|)dP.  In  addition,  if  the  Banach  space  L***  is  reflexive, 

we  have  L$  =  [3].  Hereafter  we  posit  all  Orlicz  spaces  are  reflexive. 

Let  B  be  a  Banach  space.  We  say  B  is  (i)  locally  uniformly  convex  if 


whenever  (x ,  and  (y„}“  n  are  sequences  in  the  unit  ball  of  B  with 
n  n~ I  n  n= I 

lim  ||xn+yn||  *  2,  lim||xn-yn||  =0;  (11)  uniformly  convex  if  Ve  >0  there 

exists  6  >0  so  that  for  any  x,yeB  so  that  ||x||  =  ||y||  =1,  ||x+y|| 

2(1-6)  *?>  1 1 x-y 1 1  <e.  Uniform  convexity  is  a  much  stronger  condition  than 
local  uniform  convexity.  Every  uniformly  convex  space  is  reflexive  [6], 


pp.  126-128.  For  more  about  these  sorts  of  conditions,  see  [2], 


If  B  is  a  Banach  space,  Q  cB  is  said  to  be  proximal  if  VxeB,  the 

»I^0|>,CT,  min  ||x-z|| 
zeQ 

possesses  a  solution.  If  B  is  reflexive  we  may  apply  the  Smul'lyan  theorems 
to  see  that  every  closed  convex  subset  is  proximal.  Moreover,  if  B  is 
strictly  convex  the  norm  minimizer  problem  has  at  most  one  solution.  In 
this  case  for  xeB  and  KcB  that  is  closed  and  convex  let  pJx)  denote  the 
unique  solution  to  ||x_z|| 

zeK 

The  map  pK  is  called  the  metric  projection  of  B  onto  K.  Shortly  we  will 


I 


see  that  if  B  is  locally  uniformly  convex  and  reflexive  then  for  any  closed 
convex  KcB,  pK  is  norm  continuous.  In  [5]  we  showed  that  pK  is  weakly 

sequentially  continuous  if  B  is  reflexive  and  strictly  convex. 

Let  M  be  a  separable  metric  space.  A  Borel  measurable  map  Q:  M+M  is 
said  to  be  a  round  off  map  if  Q  has  finite  range,  say  {pj,...»Pn>  =  range  Qv 

andQ(pk)=pk,  1  <k  <n.  The  set  (Q'1(p1),...,Q"1(pn)}  is  called  the 

partition  of  M  defined  by  Q.  A  sequence  of  round  off  maps  is  called 

n  ii“  i 

a  round  off  scheme  if  (i)  VxeM,  11m  dia  Q”  (Q„(x))=0;  (ii)  the  partition  of 

n**<»  n  n 

M  defined  by  refines  that  defined  by  Qn,  neH.  Note  o(Qn)  c=  °(Qn+i)* 

neH.  See  [5]  for  details  on  round  off  schemes. 

The  primary  result  of  [5]  is  contained  in  the 
Theorem  1 :  Let  $,4'  be  an  Orlicz  pair,  (0,^,?)  be  a  nonatomic  probability 
space,  M  be  a  separable  metric  space  and  (X(t):  te[0,T]}  be  a  stochastically 
continuous  process  on  (n,^,P)  taking  values  in  M.  Then  for  any  round  off 


it 


D 

a 


scheme  (Qn)n-|  on  M  and  any  increasing  sequence  of  partitions  of 


Codes 


[0,T]  whose  meshes  decrease  to  zero  we  have  for  any  Y  eL*(0,^,P) 


’/or 


L* 


A-t 


l|Et(V|Qn(X(t)):  t«Pm)-Et(Y|X(t):  tt[0,T])||  -0 

as  or,  equivalently, 

0  =  lim  f  *(|E^(Y|Q  (X(t)):  t«P  )  -E$(Y|X(t):  te[0,T])|)dP 
m,n-*-°°  u 

where  E(J)(*  \&)\  L^(Q,^,P) ->■  L^(Q,^,P)  denotes  metric  projection  and  &  is 
any  o-subalgebra  of 

III.  A  CONVEXITY  CONDITION  FOR  CONVEX 
FUNCTIONALS  ON  REFLEXIVE  BANACH  SPACES 

Let  B  be  a  reflexive  Banach  space  and  ip:  B-*-IR  be  a  convex,  norm 
continuous  map  satisfying 

lim  i|i(x)  =  +<»  . 

II  xlb°° 

Then  using  the  Smul'lyan  theorems  we  see  that  VxeB 

min  iHx-z)  (1) 

zeK 

possesses  a  closed  convex  nonvoid  subset  Lx  of  K  of  solutions.  Furthermore, 
if  ip  is  strictly  convex  each  Lx  is  a  singleton  (p(x)}.  We  call  the  map 
p:  B  -+K  the  solution  map  of  (1).  Let  6  >0;  define 
K6  x  =  tzeK :  iHx-z)  <6  +  ii>(x-p(x))}. 

We  say  ip  is  locally  uniformly  convex  if  lim  dia  Kr  *0  for  all  xeB  and 

closed  convex  K  c  B.  6+0  0,x 

Proposition  2:  If  <J>:  B-*-IR  is  locally  uniformly  convex  and  uniformly 

continuous  on  bounded  subsets  of  B,  KcB  is  nonvoid  convex  and  closed  and 

p:  B+K  is  the  solution  map  of  (1)  then  p  is  norm  continuous. 

Proof:  Let  x_  +x  in  8  and  5  >0.  Take  zeKr  ...  Then 
-  n  o  ,x 

iHx-z)  <6  +t|i(x-p(x)). 

Since  x„  -*-x  there  exists  NeN  so  that  n  >N  => 
n  — 

iMx^)  < 6+^(xn-p(x) ) . 

By  minimality 

>p(  Vp(xn) )  i  *(vz) 

<  <5  +  ip(xn-p(x) ) . 

Taking  limit  suprema, 

lim  sup  tl>(x  -p(x  ))  <  i|i(x-p(x)).  (2) 

n  '>oo  ■■  n 

Suppose  by  way  of  the  contrapositive  that  {p(xn)}“_^  is  unbounded. 

Since  xn  -*x,  is  bounded;  this  forces  {xn-p(xn)}”_^  to  be  unbounded. 

Observing  that  lim  <|>(x)  -+“>  it  now  follows  that 

1 1  x  j  |  -*>«> 

lim  sup  tp(x  -p(xj)  =  +», 
n  ->oo  n  n 

a  flagrant  violation  of  (2).  This  insures  that  (p(xn)}“_^  is  bounded. 

Suppose  that  for  some  zQeB,  p(*n)  -J-  zQ.  Then  xp  -p(xn)  x-zQ;  ^  is 
convex  and  norm  continuous  and  therefore  weakly  lowersemi continuous.  Thus 


Ip(x-zn)  <  11m  Inf  4>(x  -p(x  )) 
u  n  n  n 

<  11m  sup  <|>(x  -p(x  ))  <  \J>(x-p(x)). 
n  -*■<*>  "  " 

Next  observe  that  K  Is  closed  and  convex  and  therefore  weakly  closed.  Thus 
p(*n)eK,  neN,  p(xn)-»  zQ  ^  z0eK*  By  Zqsp(x)* 

Since  B  Is  reflexive  and  {p(xn)}*_j  Is  bounded  each  subsequence  of 

{p(*n)}™=1  must  have  a  further  weakly  convergent  subsequence.  But  the 

foregoing  discussion  tells  us  that  this  further  subsequence  must  converge 
to  p(x).  Immediately  It  follows  p(*n)  -*p(x). 

We  may  apply  the  weak  lower  semi  continuity  of  4>  to  see  that 

'Kx-p(x))  <  11m  Inf  <i>(x  -p(x  )). 

p  -*-oo  11  n 

Combine  this  with  (2)  to  get 

vp(x-p(x))  =  11m  4»(x  -p(x  )>. 

p  -**oo  M  n 

PutM  =  l+sup  1 1  x_-p(x_)  f | ;  noteM<°°.  Take  e  >0,  choose  n  >0  s.t.n<  1 
n  n  n 

and  ||  x-y  ||  <n,  ||x||  ,||y||  £  M  =»  |tp(x)-ip(y)  |  <e.  Because  xn-*-x  there 

exists  NeN  such  that  n  >  N  =«►  1 1  xn-x||  <  n.  Then  n  >_N 

Ilxn-p(xn)-(x-p(xn))||  =  llvxll  <  n 

so  n  >N  => 

| ^(xn-p(xn) )  " ip(x-p(x) )  |  <  e. 


This  forces  lim{i|>(x  -p(xJ)-4>(x-p(x.J)}  *0  and  hence  lim  <p(x- p(x  ))  * 

,  ,  „  n-^»  n  n  n  n-*®0  n 

'Hx-p(x))- 

Noting  11m  4i(x-p(x  ))  =i|/{x-p(x))  there  exists  NeN  so  that  n>N  => 

n-xxi  o 

ijj(x-p(xn))  <  6 +I|/(x-p(x)). 

But  this  says  that  for  n  >N,  p(x  )  eK.  .  Obviously  p(x)  eK.  so  for 

n  o  9  x  o  $  x 

n>N,  ||  p(x  )-p(x)||  _<  dla  K,  .  Since  lim  dia  K,  =0  we  must  have 

II  O , X  X  0  fX 

p(xn)-p(x). 

Proposition  3:  Suppose  {Kn}”_^,  K  are  nonvoid  closed  convex  subsets  of  B 

with  Kn  =  Kn+i»  neN  and  K  *  Kr.  Let  B-*-IR  be  locally  uniformly 

convex  and  norm  continuous.  Finally  denote  by  pn  the  solution  map  of  (1) 

corresponding  to  K  ,  neN  and  p  denote  the  solution  map  of  K.  Then  for  xeB, 
p„(x)-p(x).  " 

Proof:  Let  6  >0,  xeB.  By  the  continuity  of  i|>,  K„  „  is  a  relative  neighbor- 

oo  0 

hood  of  p(x)  In  K.  Noting  K  =  U  1^,  we  see  there  must  be  an  NeN  so  that 

n  >N  =>  K  (l  K.  f  0.  n=1 

”  n  0  »X 

Fix  n>N,  zeKj  xn  Kp.  Then  we  know  that  (i)  ip(x-z)  <  5 +t|»(x-p(x)); 

(11)  vp( x-pn( x) )  <_  ip(x-z) .  Assembling  the  facts,  n  > N  *> 


Letting  n 


'Hx-PpU))  <  5  +t|»(x-p(x)). 
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lim  sup  >Mx-p  (x))  £  ^(x-p{x)).  (3) 

n-*“»  * 

We  may  imitate  a  familiar  argument  in  Proposition  2  to  see  that  lpn(x)>n=1 
i s  bounded . 

Since  K  <=  K  <=  K,  neW  it  is  easy  to  see  that 
n  —  n+i  — 

4>(x-Pn(x))  >  ip(x-pn+1  (x) )  >  <J>(x-p(x)). 

With  the  assistance  of  (3)  we  obtain 
x-p( x) )  =  lim  4>(x-p  (x)). 

p  -foo  ' 1 

Pick  6  >0.  There  exists  NeN  such  that  n  >_N  => 
vp(x-pn(x) )  <  *(x-p(x))  +<5- 

We  may  now  conclude  pn(x)eKg  x»  n-^*  Since  dia  ^-*-0  as  6 -►O  we 
conclude  that  pR(x) +p(x)  in  norm.  2U 

IV.  SOME  FACTS  ABOUT  ORLICZ  SPACES 

Theorem  4:  Let  (fl.^.P)  be  a  probability  space  and  4>,'F  be  an  Orlicz  pair. 
Then  L$(fi,.^,P)  is  uniformly  convex  if  and  only  if 
V  e  such  that  0  <e  there  exists  R£  >0  so  that 


lim  inf  >  R  . 

n  ->-oo  ip((l-e)x)  c 

Proof:  This  is  the  central  result  of  [4]. 

Define  £:  l*(n,^,P)-  [0,«)  by  £<X)  -/*(|X|)dP.  Then  £  is  a  strictly 

convex  norm  continuous  functional  and  lim  £(X)  s  +°°-  Let  0<e  <1, 

llxllL$^  00 

X,YeL  (n.^.P)  and  X?«Y  a.s.[P].  Then  l|X-Y||  0.  Using  the  convexity 

of 


*(|X-Y|)  =  *(e(l  | X-Y | )  + ( l-e)(0)) 

£  e  )  +  (l-e)4>(0) 

-  e 

Integrating, 

jr*(ix-Yi)dP  <  e  jr$(i^-)dp. 

Now  suppose  ||  X-Y  |j  q<  1 .  Then 
4*<|*-Y|)dP  <  iix-yhl4 

-  I|x-y||l#*(D, 

amply  demonstrating  the  global  uniform  continuity  of  £. 

We  will  not  address  the  question  of  the  local  uniform  convexity  of  £ 
here.  The  hypothesis  of  Theorem  4  or  possibly  something  similar  might 


guarantee  the  local  uniform  convexity  of  £. 


V.  A  CONVERGENCE  PRINCIPLE  FOR  A 
GENERALIZED  CONDITIONAL  EXPECTATION 

Definition:  Let  (ft.^.P)  be  a  probability  space,  f  be  a  sub  o-algebra  of 
&  and  be  an  Orlicz  pair.  For  XeL^fl.^.P) ,  we  define  E^Xl^)  to  be 
the  unique  solution  to  the  variational  problem 


min  { /*(|X-Z|)dP:  Z€L$(Q,^,P) } . 

Recall  from  [5]  that  if  $(x)  =  for  x^O,  then  E(j)(*|.jF)  is  ordinary 
conditional  expectation  given 

Remark :  If  the  functional  X  J*$>(|X|)dP  is  locally  uniformly  convex 
then  E(J)(*|,^')  is  norm  continuous  on  L^Q.-^.P) . 

Theorem  5:  Let  P)  be  a  probability  space,  }“,  be  an  increasing 

00  ll“  I 

family  of  sub  o-algebras  of  &  and  put  &  =  V  &  .  Suppose  4>,i'  is  an 

n=l  n 

Orlicz  pair  so  that  X  —*■  f  4>( } X |  )dP  is  locally  uniformly  convex.  Then  for 
XeL*(Q,^,P),  Q 

E^XldF)  E$(X|dT)  as  n  -►«, 

or  equivalently. 


lim  /  «(|E _)-L(X|^)|)dP  *  0. 
n  ■>o°  •'ft  * 

Proof:  This  is  an  immediate  consequence  of  Proposition  3.  (JED 

We  may  now  proceed  as  in  [5]  to  get 
Theorem  6:  Let  P)  be  a  probability  space,  {X( t ) :  te[0,T]}  be  a 
process  on  P)  taking  values  in  a  separable  metric  space  M  and  be 

an  Orlicz  pair  with  X  Jf^$(|X|)dP  locally  uniformly  convex.  Then  if 

is  a  round  off  scheme  for  M  and  {Pm)^_i  is  an  increasing  sequence 

of  partitions  of  [0,T]  whose  meshes  decrease  to  zero,  we  have  for 

YeL<I,(Q,^',P) 

E,(V|Q„(*(t)):  t€Pm)  E4(Y|X(t):  t«[0,T]) 
as  m.n-^oo,  or  equivalently, 

1im  lE^tYlQ  (X(t)):  teP  )  -E.(Y|X(t):  te[0,T])|)dP  =  0. 

m,n-«°  «  1,1  v 

VI.  CONCLUSIONS  AND  A  PARTING  SHOT 
AT  EARLIER  WORK 

The  abstract  principle  developed  in  Section  III  subsumes  the  Banach 
space  principle  developed  in  [5].  Simply  take  the  functional  to  be  the 
norm. 

We  have  enhanced  the  feasibility  of  studying  the  operators  E^  numeri¬ 
cally  in  two  ways.  First,  knowing  that  E^  is  continuous  assures  us  that 


£- 


±a. 


it  will  tolerate  small  L^-perturbations.  Second,  the  result  of  Theorem  6 

shows  us  we  may  approximate  E(J)( Y | X( t) :  te[0,T])  by  E$(Y|Qn(X(t)) :  tePm). 

Since  o(Qn(X(t)):  tePm)  is  finite,  calculation  of  E$(Y|Qn(X(t)):  tePm) 

consists  of  solving  a  finite  dimensional  nonlinear  optimization  problem.  A 
large  body  of  knowledge  now  exists  about  the  numerical  solution  of  such 
problems. 
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